The design of clinical trials is at the center of new drug development: trials must be designed in an optimal manner if they are to yield quality answers to the research questions of interest. However, the biopharmaceutical industry’s organizational approach to drug development does not tend to utilize design best practices and apply them to either the design of individual trials or clinical development programs. The reasons for this inherent paradox — lack of good design in organizations that have the purpose of facilitating optimally designed clinical trials — are twofold. Firstly, R&D organizations tend to be centered around cross-functional project teams, typically formed on the basis of technical or therapeutic expertise, not on well-practiced design skills and processes. Secondly, these teams are typically not incentivized to foster open approaches to design or to use structured design methodologies that lead to high quality and performance.

I shall explore and advocate a different approach that is grounded in best practices from other industries, one in which structured design methodologies play a pivotal role. By linking design best-practice with a structured, information-driven approach, biopharma companies can make necessary changes to how they approach new drug development, and empower their experts with tools to make better design decisions.

New Drug Development: Elements and Challenges of the Current Paradigm
Several design elements are fundamental to the conduct of clinical trials as part of a new drug’s clinical development program. These include the structure of the study design, the experimental methodology used, the operational execution plan, and ultimately, the statistical analysis and interpretation of the results.

Designing Clinical Trials
ICH Guideline E8, “General Considerations for Clinical Trials,” reads as follows:¹
Clinical trials should be designed, conducted, and analysed according to sound scientific principles to achieve their objectives; and should be reported appropriately. The essence of rational drug development is to ask important questions and answer them with appropriate studies. The primary objectives of any study should be clear and explicitly stated.

Piantadosi² echoed this guidance, while also reminding us of the importance of biological considerations. The ultimate goal of new drug development is to produce a biologically active drug that is acceptably safe (with “acceptable” being defined in conjunction with efficacy as a suitably favorable benefit-risk balance), well tolerated, and useful in the treatment or prevention of patients’ biological states that are of clinical concern.³ Such biological activity is the result of the positive interaction between the drug and a drug target within the body, and the resultant cascade of biological signals. This should be assessed by easily testable scientific hypotheses, with clear measurable clinical outcomes. Piantadosi therefore commented as follows:

The most critical and difficult prerequisite for a good study is to select an important feasible question to answer. Accomplishing this is a consequence primarily of biological knowledge. Conceptual simplicity in design and analysis is a very important feature of good trials. Good trials are usually simple to analyze correctly.

Interpreting trials correctly is intimately linked to making good decisions; the information provided by a trial is the basis for rational decision-making. The results obtained, however, are predicated on what is designed upfront. Hence, “designing with the end in mind” is critical to determining the content and the quality of the information that will be available for the next development decision milestone.

Experimental Methodology, Operational Execution, and Analysis & Interpretation
Experimental methodology and operational execution planning are concerned with all aspects of the implementation and conduct of a study; the goal is to collect optimal quality data that will provide clear answers to the research question of interest (see Turner⁴ for a conceptual discussion of this topic — no complex statistical equations are presented). The key research questions should specifically address the benefit-risk-value propositions for decisions relevant to the particular phase of drug development.

Challenges of the Current Paradigm
The major challenge facing all biopharma companies is the financial risk inherent in developing new drugs. The cost of developing a new medical entity is very high relative to the probability of achieving a successful market entry for the product. First, consider the cost component. Cost estimates of new drug development provided in the literature are remarkably variable. Morgan et al.⁵ performed a systematic review and found that the highest estimate was nine times higher than the lowest estimate. An often-cited figure from the Tufts University Center for the Study of Drug Development is more than $1 billion.⁶ Perhaps more importantly, the success rates for investment are highly variable across different biopharma companies. On average, it is only about 8% for a compound entering man to reach the market, which equates to a 92% failure rate. Even compounds entering Phase III trials only have a 50-50 chance of reaching the market successfully. These statistics highlight both the risk and the expense of iterative poor outcomes.

Given these constraints, biopharma companies must either reduce R&D expenditure, increase the probability of success, or figure out more cost-effective ways to bring drugs to market. Incremental cost-cutting can only go so far before R&D departments no longer have the bandwidth to move compounds effectively through clinical development programs. Therefore, it becomes critical to increase the probability of success for the molecules in a company’s pipeline. One of the few strategies the industry can employ is to incorporate better planning and design principles into the various stages of new drug development. For the rest of this article, I shall focus on this strategy, the employment of structured design methodologies, and lessons learned from other industries.

Given what is at stake with each program or trial design, why is there so little attention paid to how design activities are conducted in biopharmaceutical R&D organizations? The main challenge to design activities in the current drug development paradigm can be found in suboptimal organizational practices. The organizational approach of many biopharma companies’ development activities is centered around project teams that are typically formed on the basis of technical or therapeutic expertise, rather than on well-practiced design skills and information. Moreover, these teams are not typically incentivized to foster open approaches to design or to use structured design methodologies that lead to high quality and performance. Consequently, there is considerable variability in the quality of the programs and trial designs produced by project teams, especially if these teams are set up to drive cross-function execution rather than integrative “design thinking”! A team focused on execution will be more interested in meeting a “first patient in” deadline than the quality of the upfront design, even if the result of the poor upfront design is less efficient clinical trial execution or failed outcomes, i.e., failing to achieve the scientific goals of the study.

A more systematic approach to design that is grounded in best practices from other industries could result in improved success rates and more cost-effective, timely, clinical trial execution. To realize these improvements, however, the biopharma industry must recognize that design can be conducted as a structured process, and that design tools can be developed to facilitate such a process.

Re-examining the Design Process
As discussed previously, the current paradigm in biopharma drug development is structured to a large extent around subject matter expertise that is assembled in project teams. Consequently, design activities are conducted as an “expert teaming” process, not as a true design process. While it is certainly true that expertise is a critical component of the process, exploring methodologies used in other industries clearly reveals that “design thinking” can be performed in a structured way. 

The utilization of structured design methodologies involves four key steps: “Define – Integrate – Prototype – Crystallize.” Defining the issue to be addressed, i.e., the key questions for the next decision, focuses the design team and allows it to develop clear, measurable objectives. “Integrate” is the activity by which the design team brings together all relevant information. Quality design is predicated upon good information. “Good information” has several sequential requirements:

• access to multiple, reliable sources of data
• proper tools for objectively analyzing and modeling these data, and
• the application of the knowledge gained from the data to the design process.

The “Integrate” step is all about bringing together this information and applying it.

While access to quality, timely information is one of the keys to design success, it is equally important to identify what is not known and what assumptions are being made by the team. This allows the next critical step, “Prototyping,” to take place. This is the step in which the team ‘plays’ with the different assumptions to develop different outcome scenarios, models these outcomes based on the information available, and understands options and trade-offs. Doing this enables the team to choose the option best suited to address the issues it has defined, thus “Crystallizing” their final design decision.

In the current biopharma development paradigm, implementation of this approach is challenging for project teams, since each individual trial design — and by extension, the design nature of every development program — is complex and fraught with assumptions and contingencies. Furthermore, access to robust realtime information is usually limited, making it difficult to develop and understand multiple scenarios. Therefore, a more systematic and transparent approach could achieve far greater clarity of options, and hence a better probability for successful outcomes. This can be achieved by adopting one of the central tenets of design thinking: integrating known information with determination of unknowns (e.g., hypotheses, assumptions, risks). Doing so allows the design effort to focus directly on addressing these unknowns, while minimizing the inadvertent introduction of bias. For various potential reasons — including lack of access to information, time, resources, or good processes — the biopharma industry is not efficient with integration and prototyping, which makes it difficult to test designs and influence outcomes. A good structured design process, utilizing a proper design tool, could markedly facilitate a company’s ability to visualize and explore data-driven options in a transparent, open way, increasing the probability of better design decisions. This may also help companies reduce the complexity of their designs, decrease trial costs, and ultimately end up with clearer decisions at the end of each trial.

Lessons Learned from Other Industries
A recent study of leading global companies spanning several diverse industries examined what constitutes “best practice in design.”⁷ Commissioned by a large biopharma client and conducted by the Hay Group, an international management consultancy, the study revealed that companies most admired for their design practices share many of the same principles:

1. Mindset: Design is a disciplined, methodical approach embedded in the organization; there is a healthy tension between innovative and creative thinking and disciplined behavior and design processes.
2. Open innovation: Models are vital to generate both collaboration and ‘competition’ among internal and external partners.
3. Purpose: Design teams are created and exist to drive business outcomes.
4. Metrics: Companies use a few key metrics to assess design quality and performance, focused on activities that drive value.
5. Knowledge Management: Ideas, technology, and projects are stored, re-cycled and re-used; companies excel in at least one of these dimensions and nothing is thrown away.
6. Execution: More so than strategy or structure, design processes and practices are viewed as having the greatest impact on execution.
7. Leadership: Leadership plays a central role in defining expected outcomes and conditions for success and takes a portfolio approach to design, while fostering “bottom up” generation of innovative approaches.

How can these lessons be applied to our industry? Many, if not most, of these practices are not evident in the design practices of most biopharma companies. The ‘failure statistics’ noted previously have likely roots in a development path that does not use the best design approaches.

The contrast in the organizational/cultural mindset of the biopharma industry and companies that successfully embrace design thinking is well exemplified by consideration of a company that excels in this regard. One prominent European automaker, widely regarded by both customers and its peers for its outstanding designs, has created a culture of design thinking, one in which multiple internal teams compete but do so in a collegial spirit.⁸ Each team is given a design remit, i.e., they have to produce the best possible design within the specifications that they are given. However, the final design choice is rarely the result of the work of a single team. Each team is incentivized to produce the best possible results it can within their initial remit. Even when one team’s design is well on the way to being chosen, the best elements from competing designs are brought together, and hence every member of every team has the possibility of influencing the final design in a competitive — yet ultimately collaborative — manner.
This approach yields multiple possible options for the final design, which in turn yields higher quality and a more focused, robust ultimate result.

Understanding Failure
Most biopharma design teams are incentivized and plan for the success of their compounds. However, as noted previously, at least nine out of 10 compounds entering man will not reach the market. There are various reasons why products fail during clinical trials; these can broadly be attributed to one or more of the following areas:

1. Bad compound: Although advances in discovery technology have reduced the number of “bad” compounds brought into development, there are still cases in which the investigational drug does not have the proper pharmaceutical properties for human use, has predictable side effects that limit its therapeutic window, or simply fails to hit its biologic target.
2. Translational failure: Translational research, though much improved over the past two decades, is still an inexact science with many unknowns, and researchers often struggle to make the appropriate linkage between animal biology, human biology, and hard clinical outcomes.
3. Unexpected safety findings: Clinical research will always contain the unknown and often produces safety signals that simply could not have been anticipated until large populations are studied.
4. Poor design: Poor design can be attributed to a number of reasons, including a desire to save costs and time, and designing based on aspiration rather than pragmatism and realism. However, the three most common root causes of poor design are a lack of clear definition (design remit), a lack of full information integration, and a lack of prototyping. Human bias (especially in a team process) factors into each of these steps, as well as overall design decisions,⁹ and can be a major factor in design outcomes unless addressed objectively and transparently.

Of these reasons for failure, only two are realistic levers a biopharma company can pull to improve the chances of getting a new molecular entity to market: (1) selection of the compounds taken from nonclinical to clinical development programs; and (2) better design processes and decisions. The two other factors — translational failure and unexpected safety findings — are far less predictable.

Given the critical role design can have on program outcomes, it is essential to understand the results of poor design (rather than “blaming the molecule” for bad outcomes).  Poor design can result in research that is not focused on answering the appropriate questions concerning a particular upcoming investment decision. Studies may be statistically underpowered to achieve the trial’s scientific objectives, or over-powered relative to the real risk, resulting in great excesses in cost and time, with either resulting from poor design assumptions. There can be a lack of proper dosing information, or a failure to identify the optimal subject target population appropriately, both of which are key design considerations.

Such scenarios can result in ambiguous trial outcomes, and, in the extreme case, not provide the information necessary to determine whether it was the drug that didn’t work or the design that didn’t work. When this happens, drug candidates might be killed or progressed inappropriately (at great expense vs. the true probable outcome). Additionally, companies might also need to repeat studies to obtain clarity as to whether to continue or to terminate the development program. To illustrate the problems of ambiguity, consider a Phase III trial with a marginal outcome (e.g., a p-value of 0.07): one of the company’s possible interpretations of this result could be that the drug is actually working, but that the design chosen was insufficiently rigorous to facilitate achieving a statistically significant result that would allow the drug’s registration application to be successful.

The Relationship Between Design and Decision-making
So far, we have discussed the benefits of a good design process and the consequences of design failure. By “designing with the end in mind,” the design should create value by meeting the needs of a program at any given stage of development. This is easier said than done, but good planning and design can lead to two key opportunities: (1) a higher probability of achieving a trial’s scientific objectives, and (2) enhanced operational execution that decreases both costs and time to market. Ultimately, companies should be able to make better, quicker, and more finely-tuned investment decisions, meaning that only drug candidates with the most potential for success are progressed, while reducing investment costs and optimizing return on investment.

Figure 1 is a stylized representation of decisions that need to be made at each major milestone in the drug candidate’s lifecycle. The figure is based on a well-known design paradigm called “The Cone of Uncertainty.” The idea is that, at each milestone, a design is structured to reduce uncertainty about the product for the next milestone decision-point. In the biopharma industry, this means that a quality design focuses on gathering just the required information needed about the benefit-risk-value (BRV) proposition of an asset, while reducing the unknowns concerning biology, safety, and efficacy. Every element in the process is focused on the next milestone and ultimate target profile for the molecule to obtain better information for the next investment decision. It is this design focus, supported by the appropriate methodology and tools, that stands the best chance of altering the industry’s struggling ROI paradigm.

The biopharmaceutical industry stands to gain considerably from embracing and implementing a culture of design thinking, thereby incorporating best design practices across the drug development lifecycle from discovery to commercialization. Design skill needs to be embedded in biopharma organizations, and design teams supported by good design practice, access to information, and appropriate design tools. Design activities should then be assessed on how they create value and provide the additional information required to enable knowledge-based investment decision-making. 

References

1. ICH Guideline E8. Available at: http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E8/Step4/E8_Guideline.pdf (Accessed 12th March, 2013)
2. Piantadosi S. Clinical trials: A methodologic perspective, 2nd Ed. Hoboken, NJ: Wiley-Interscience. 2005.
3. Turner JR. New drug development: Design, methodology, and analysis. Hoboken, NJ: John Wiley & Sons. 2007.
4. Turner JR. Key statistical concepts in clinical trials for pharma. New York: Springer. 2012.
5. Morgan S, Grootendorst P, Lexchin J, Cunningham C, Greyson D. The cost of drug development: a systematic review. Health Policy. 2011;100(1):4-17.
6. Tufts Center for the Study of Drug Development. Outlook 2010. Available at: http://csdd.tufts.edu/_documents/www/Outlook2010.pdf (Accessed May 29th, 2013).
7. Hay Group. Best Practices in Design and Sustainable Innovation. 2009.
8. Sax, FL. Clinical trial planning & design: Can better design save biopharma? Available at: http://www.quintiles.com/Library/Article.aspx?articleID=27730 (Accessed 28th May, 2013)
9. Kahneman D. Thinking, Fast and Slow. New York: Farrar, Straus, and Giroux. 2011.

---

Rick Sax, MD, is global head of the Center for Integrated Drug Development, Quintiles.  He can be reached at [email protected].